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Abstract 

The robustness and power of 9 strategies for testing the differences in groups’ regression slopes 
were assessed under nonnonnality and residual variance heterogeneity. For the conditions 
considered, the most robust strategies were the trimmed and Winsorized slope estimates used 
with the James second-order test, the Theil-Sen slope estimates used with James, and the Theil- 
Sen estimates used with percentile bootstrapping. The use of Theil-Sen slope estimates produced 
more powerful tests than the use of trimmed and Winsorized slopes. 

Key words: Slopes, least squares, Theil-Sen, robust regression, James second-order, 
nonnonnality, residual variance heterogeneity 
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Introduction 

The issue of group differences across an individual difference variable (X) has been 
considered in many fields of social science (Aguinis & Pierce, 1998; Cronbach & Snow, 1977; 
Dance & Neufeld, 1988; Hunter, Schmidt, & Hunter, 1979; Seligman, 2002). The typical 
approach of strategies evaluating group differences across X involves fitting regression lines that 
predict outcome Y from X in separate treatment groups and then conducting a significance test 
for the homogeneity of the groups’ regression slopes. In this study, strategies for comparing 
slopes are evaluated under conditions of nonnormality and residual variance heterogeneity. 

The slope test strategies considered in this study are approaches to estimating the 
following model, 

Y v= Poj+PA+e,, (1) 

where outcome Y for individual i (= 1 to AO in group j (= 1 to J) is a linear function of a 
continuous X, p 0j and p^ are the population intercepts and slopes of the regression line for each 

of J groups, and the e ;j are the residuals. The strategies for assessing differences in the p, : ’s 

reviewed below are most easily understood in tenns of alternative expressions of (1). When 7 = 
2,(1) can be expressed as, 

Y u = Po + Pi x u + P 2 Gij + P 3 X ij G ij + Sy , (2) 

where G ;j is a dichotomously-coded group membership variable. A more general matrix version 
of (1) and (2) is, 

Y = xp + e , (3) 

where Y is an Af-by-1 column vector, X is an N-by-K design matrix corresponding to the K /? ’s 
(including a column of 1 ’s for estimating / 3 0 ), fi is a K- by-1 column vector of f3 ’s, and e is an 
X-by-l column vector of residuals. 
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Standard Slope Estimation and Slope Test 

This section presents the standard least squares slope test and its technical details. The 
technical details provide a basis for understanding the alternative and robust versions presented 
in the later sections. The standard slope test uses least squares estimates of thep ’s (i.e., p ’s) that 

minimize the sum of the squared residuals, e'e = ( y - X[l ) (V - xp) . Because e'e is a convex 

function of p , it can be minimized by differentiating with respect to p , setting this derivative to 
zero, and solving for p , resulting in the closed form solution, 

p^x'xy'x'Y (4) 

I<X. -x )(y -y,) 

Equivalently, group f s slope can be estimated as, (L = 111 J „ , (5) 

unj 



where x ■ j and y , are the means of X and Y in group j. 

The standard test for assessing the differences of J slopes is an F test, 
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where a 2 = — 

N r 2 



and p 



1. Standard 
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j 1 ln j 



is the variance-weighted common slope 



(Myers & Well, 1995, p. 421-422). (6) is evaluated on an F distribution with J-l and N-2J 
degrees of freedom. With J = 2, a t-test of p, in (2) that is equivalent to the F test in (6) can be 
conducted by obtaining the standard error of P 3 as the square root of one of the diagonal 



„ r 

elements in the variance-covariance matrix of B , (X‘X) 1 , and evaluating — — on a t 

N-K SE(p 3 ) 



distribution with N - K = N - 4 degrees of freedom. The referencing of the standard test statistics 
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to F and t distributions is justified when the data meet particular assumptions, namely that the e y 

are normally and independently distributed with equal variances across the J groups. 

The standard methods for estimating and testing slopes are problematic when data are 
nonnonnal and residual variances are heterogeneous (Alexander & Deshon, 1994; Conerly & 
Mansfield, 1988; Conover & Iman, 1982; Deshon & Alexander, 1996; Dretzke, Levin & Serlin, 
1982; Headrick & Sawilowsky, 2000; Klockars & Moses, 2002; Overton, 2001). When 
distributions exhibit heavy-tailed nonnormality, extreme scores occur more often than when 
distributions are normal, increasing the variability of the estimated slopes, reducing the estimated 
standard errors, and making the standard test excessively liberal. When groups’ residual 
variances and sample sizes differ, the standard test’s pooling of groups’ residual variances, 

j - 2)cr , is problematic, making the standard slope test either liberal or 

conservative, depending on which of the larger and smaller groups has the larger or smaller 
residual variance. The inaccuracy of the standard test is disturbing, given that nonnormality and 
residual variance heterogeneity appear to be common in actual data (Aguinis, Peterson, & Pierce, 
1999; Micceri, 1989). What follows are detailed definitions of slope test strategies that may 
outperform the standard test when distributions are nonnonnal and residual variances are 
heterogeneous. 

Slope Tests for Nonnormal Data: Central Tendency Strategies 

X< x - x •><>'■->'> 

Two approaches to slope estimation view group j’s slope in (5), (L = — . , , 

2>i.i - x ,i> 

i in j 

as a central value of the slopes that can be created from pairs of observations in the data, 

(y r - y f .) 

b, „ = — - — — i^ i 1 x x., , and then try reduce the influence of the extreme observations on the 
' • : (x, -x,,) u 1J ’ 

central value. These central tendency approaches define extreme observations in terms of both A 
and Y, so that the screening of extreme observations caused by nonnormality could potentially 
address slope estimation problems such as leverage (observations that are extreme on X ), 
discrepancy (observations that are extreme with respect to the regression line), and outliers on Y. 
One popular strategy is the Theil-Sen slope estimator (Ebrahem & Al-Nasser, 2005; Sen, 1968; 
Theil, 1950; Wang, 2005; Wilcox, 2004; Wilcox & Kesehnan, 2004). The Theil-Sen estimate is 
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the median of the slopes that can be computed from the Nj(Nj-l)/2 pairs of observations in the 
data. Percentile bootstrapping methods can be used to test for differences between groups’ Theil- 
Sen slopes (i.e., draw 599 random samples with replacement from the J = 2 datasets, compute the 
differences in Theil-Sen slopes in each of these datasets, and determine if the middle (l-a)% of 
the 599 slope differences contain zero; Wilcox, 2005). 

A less-familiar alternative to the Theil-Sen slope estimate is the application of the 
trimming and Winsorizing strategies that are typically proposed in tests of mean differences to 






(Guo, 1996; Luh & Guo, 2000). To obtain trimmed and Winsorized estimates of slopes 



( x u - x rj) 

and their variances, rank order the x’s in each of the J groups, xy< X 2 j. . ..< xuj. When the number 
of observations in group j is even (A) = 2mj), one can consider m, independent slope estimates, 



l»i +m jj»ij 



_ (y i+mj j - y,j) 

< X n,. - X ,,> 



( 7 ) 



When the number of observations in j is odd (N } = 2mj + 1), a pooling is done so that 
observations y 2m j and y 2m +lj are pooled, x 2m ( and x 2m +lj are pooled, and y 2m j and x 2m ■ are 

replaced by y 2mjJ = (y 2mjj + y 2m , +Ij )/2 and x 2mjj = (x 2mjj + x 2m . +Ij )/2 . 

The trimming and Winsorizing are done for each of the j slopes and standard errors. Let 
gj = ymj where y represents the proportion of observations to be trimmed from each tail of the 
ordered distribution of b Uj ,Z = 1 to m^b, (j < b, 2j < ...b, m j . Let hj = mj-2gj be the effective sample 



size after trimming. 



The trimmed mean slope is computed as 



m rgj 

Z K 



(8) 



Winsorized slope observations are obtained by, 
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The variance of the trimmed mean slope is computed as a function of the Winsorized variance, 



' bwj 
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( 10 ) 



To assess the differences in trimmed slopes, replace the in (6) with b Lj , the 
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Vi 7 Z h j( h j -^bwj » and the Z( x ,-X,)' with h . These 

j i in j - 1 

V J 7 

replacements to (6) cause the standard test of slope differences to resolve into the F test for 
independent trimmed means with J - 1 and Z b j " J degrees of freedom, 
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Slope Tests for Nonnormal Data: Minimum Maximum Likelihood Type (MM) Regression 
In minimum maximum likelihood type (MM) regression (Yohai, 1987), extreme 
observations are addressed in the minimization process used to estimate the regression line. 
While the standard slope estimation process is based on minimizing the sum of all squared 
residuals, the robust regression paradigm views the least squares approach as one of several 
possible functions, £ , of the scaled residuals that could be minimized, 



zz# 



Yij - Poi x iji -YjzPij 



=zz* 



( g A 
b ij 



( 12 ) 



Some choices of £ can produce p estimates that outperform the standard test’s p ’s in terms of 
their breakdown rates (i.e., the smallest percentage of contaminated observations needed to 
render p useless). One popular c (SAS Institute, 2003) is the Tukey weight function, 



5 




if S < K, 



(13) 



£(s) = 1 



3 
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otherwise. 



In (13), k is a constant selected to obtain desirable properties. A k value of 3.44 results in 
parameter estimates that are 85% as efficient as least squares estimates when the data are nonnal 
(Holland & Welsh, 1977). When data contain outliers that are discrepant with respect to the 
regression line, k defines a range around which the observations outside of the range have 
reduced contribution to the slope estimates. 

The search for p 's that minimize (12) is similar to the standard test, in that p k 's are found 



such that the derivatives of (12) with respect to the p k 's are zero, 




= 0,k = 1 to K . 



Unlike the least squares estimation used with the standard test, with MM regression there are no 
closed-fonn solutions to minimizing (12). Appendix A presents an outline of the three-stage MM 
algorithm for estimating the p k 's. 

The kinds of nonnormality for which MM might be especially useful are probably 
situations with outliers that do not mask themselves by exerting heavy influence on the 
regression line. Many of the steps of the MM estimation process are analogues to the standard 
test’s estimation, including the use of least squares estimation used in the least trimmed squares 
(LTS) starting values, the computation of the P k ’s (Equation A4 in Appendix A is a weighted 
version of Equation 4), and the computation of the MM standard errors ( W in Equation A5 is a 



weighted version of (X‘X) in 



(X‘X) '). The relatedness of MM computations to the 

N-K 



standard test’s computations suggest that both procedures would do well with nonnal 
populations, while MM should outperform the standard test when there are outliers on Y 
(Anderson & Schumacker, 2003). 



Slope Tests for Heterogeneous Residual Variances 

Alternative parametric significance tests have been developed by Welch (1938), James 
(1951), and Deshon and Alexander (1994) to test for slope differences when residual variances 
are unequal. All three tests avoid the standard test’s pooling of groups’ residual variances in (6) 
Comparative research has shown that the three parametric alternative tests perform similarly in 
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terms of robustness and power (Deshon & Alexander, 1996; Luh & Guo, 2000, 2002), so this 
study focuses solely on the James second-order test, which is slightly better than the Welch and 
Deshon and Alexander tests in terms of power and robustness to nonnormality. 

The steps of the James second-order test are as follows: 

1 . Define a James weight, wy , based on each group slope’s standard error, 



wj, = 



j 



(14) 



2. Define a variance-weighted common slope as, 

/f -Zwj //.. (15) 

j 

3. Define the James’ test statistic as, 

(Azfl 

A ' 

(16) 

4. Evaluate the significance of the James’ test statistic by determining if it exceeds a 
critical value that is based on a mixture of chi-square distributions (Appendix B). 

Hybrid Slope Tests for Nonnormal Data and Heterogeneous Residual Variances 

Slope test strategies are not necessarily robust to problems for which they were not 
directly designed. The parametric alternative strategies that were designed to address residual 
variance heterogeneity have documented problems with nonnonnal data (Deshon & Alexander, 
1996). The slope tests that have been proposed for nonnormal data do not directly address 
residual variance heterogeneity. An important area of research assesses so-called hybrid slope 
tests that may be robust to several assumption violations by use of nonnonnality-robust group 
slopes and standard errors with parametric alternative tests that avoid the pooling of 
heterogeneous residual variances. 



James = ^ 

j 
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Recent research on hybrid slope tests has considered using standard slope estimates and 
standard errors or trimmed slope estimates and Winsorized standard errors with skew-corrected 
versions of parametric alternative tests (Luh & Guo, 2000, 2002). The use of the trimmed slopes 
and Winsorized standard errors with parametric alternative tests like the James test is 
straightforward, with groups’ degrees of freedom calculated as v- - h ( - 1 rather than as N ( - 2 . 

Luh and Guo also transformed the test statistics of the parametric alternatives to eliminate the 
effect of skewness (Hall, 1992; Johnson, 1978). For example, the proposed transformation for 
skewness for the James second-order test statistic from (16) is, 



James _TT = ^ 



(PrF) 






3 3 

Tx./, 



[Jhzll 

N ,A 



\ 2 



(6N.)i 



(17) 



where yk and yk are the sample skews of X and s in group j. Luh and Guo’s studies showed 
that their hybrid strategies were robust to both nonnormality and residual variance heterogeneity. 

This Study 

This study directly compares the standard, MM, and Theil-Sen tests, extending the 
previous comparisons based on estimating one slope that have given recommendations for MM 
regression over the standard test (Anderson & Schumaker, 2003) and for Theil-Sen over MM 
regression and the standard test (Wilcox & Keselman, 2004). The comparison of the trimmed 
and Winsorized slope test with the Theil-Sen and MM tests has not been considered in previous 
studies, and it allows for an evaluation of some trimming (trimmed and Winsorized) with the 
most extreme trimming possible (Theil-Sen). 

This study also extends Luh and Guo’s (2000, 2002) work, first by separately evaluating 
the trimmed and Winsorized slope test and the skewness transfonnation of the James test 
statistic. Because the accuracy of slope estimation has more to do with the heaviness of the 
distribution’s tails rather than its skew (Klockars & Moses, 2002), the test statistic 
transfonnation ought to have a smaller impact in conecting for nonnormality than the trimmed 
and Winsorized, Theil-Sen, and MM tests. Finally, Luh and Guo’s efforts to form hybrid slope 
test strategies that are robust to both nonnormality and residual variance heterogeneity are 
extended to consider hybrid slope tests based not only on integrating the trimmed and 
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Winsorized tests and the skewness transfonnation with James second-order test, but also the MM 
and Theil-Sen tests. 



Method 

A simulation study was conducted to investigate the relative robustness and power of 
nine slope test strategies for comparing two groups’ slopes. The robustness of the strategies' 
Type I error rates was assessed across 64 conditions where the population slope difference 
between the groups was zero. The 64 conditions featured a crossing of eight degrees of 
nonnonnality in Y with four group sample size combinations and two degrees of residual 
variance heterogeneity. For each of the 64 conditions, the slope test strategies' rejection rates for 
the null hypothesis of population slope differences of zero were computed by conducting their 
significance tests in 10,000 randomly generated datasets. Strategies' power rates were assessed 
across the four group sample size combinations by computing their rejection rates in 10,000 
random datasets generated with nonzero population slope differences, nonnal distributions, and 
equal residual variances. 

Nine Slope Test Strategies 

Five stand-alone slope test strategies and four hybrids of the five strategies were 
evaluated: 

1. Standard: The standard F-test of slope differences in (6). 

2. James: The James parametric alternative test in (16). 

3. MM: Significance testing of the p, in (2) based on MM estimation with the default 
settings in SAS PROC ROBUSTREG (SAS Institute, 2003). 

4. TW: The trimmed and Winsorized slope test in (1 1) using 10% trimming. 

5. TS: The Theil-Sen estimator with percentile bootstrapping for the significance 
testing. 

The following four hybrid strategies were also considered: 

6. James-TT: The James procedure with the Johnson’s one-sample /-statistic 
transformation for skewness in (17). 

7. James-MM: The James procedure using MM slope estimates and standard errors. 
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8. James-TW: The James procedure using 10% trimmed slope estimates and Winsorized 
standard errors from Luh and Guo (2000). 

9. James-TS: The James procedure using the Theil-Sen slope estimates and the standard 
deviations of 599 bootstrapped Theil-Sen estimates from Strategy 5 for the group 
slopes’ standard errors. 

Y’s Distribution 

Y s distribution was manipulated to produce eight shapes, including a normal shape (skew 
= 0, kurtosis = 0), and seven other shapes with various degrees of skews and kurtosis (Table 1). 

Y s distribution was directly manipulated because this produced situations with high leverage 
points, that is, situations that affected the slope estimation strategies to a greater extent than other 
forms of nonnormality (i.e., in e and/or X). The influence of these particular fonns of 
nonnonnality on the slope test strategies as compared to other forms of nonnormality is 
addressed in the discussion. 

Table 1 



Y’s Generated Shapes and the Fleishman Constants Used to Generate Them 



y’s 

skew 


Y s 

kurtosis 


A 


b 


c (= -a) 


d 


0 


-1.15 


0 


1.34 


0 


-0.132 


0 


0 


0 


1 


0 


0 


1.2 


1.11 


-0.340774 


1.095718 


0.340774 


-0.080735 


1.6 


2.86 


-0.418206 


0.975506 


0.418206 


-0.06113 


0 


3 


0 


0.782 


0 


0.068 


0 


6 


0 


0.66269 


0 


0.101888 


0 


9 


0 


0.573 


0 


0.126 


0 


12 


0 


0.498 


0 


0.145 
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Variance Heterogeneity 

The two considered residual variance ratios for the groups were 1/1 and 3/1. For 
conditions of unequal sample size, the residual variances were directly and inversely paired with 
the treatment group sample sizes. 

Sample Sizes 

Treatment groups of 20 and 40 participants were used. The conditions of unequal sample 
size used 20 participants in one group and 40 in the other. 

Data Generation Method: Robustness 

The following data generation method was used to create X and Y variables of desired 
distributions and variances with equal slopes in the two groups. 

1 . N values of one standard nonnal variate, Z, were generated, where N is the total 
sample size in two groups. 

2. Y was created as a transfonnation of Z using Fleishman’s (1978) method for 
generating nonnormal variables: 

Y = a + bZ + cZ 2 + dZ 3 (18) 

The constants (a, b, c, and d) and resulting distributions are listed in Table 1. 

3. An error variable for X ( e ) was generated as a standard normal variate. X’s degree of 
nonnonnality was a compromise between T s nonnonnality and s ’s nonnality. 

4. Desired numbers of Ys and e s were randomly assigned to treatment Groups 1 and 2. 

5. X was created as a function of Y and s : 




where p 7 is the desired XY correlation for treatment group j. 

6. Yjj was multiplied by a number, a Yj , that resulted in a desired standard deviation for Y 
in the jth treatment group and, in conjunction with p ; , a desired residual variance. The 
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values of a Yj and p, for the two groups achieved a particular residual variance ratio 
(Table 2), while keeping the slopes equal in the two groups. 



Table 2 



Correlations and Standard Deviations Used to Create Levels of Residual Variance 
Heterogeneity 



Residual 


Pi 


ClYl 


p2 


CTy2 


variance ratio 










1/1 


0.5 


1.0 


0.5 


1.0 


1/3 


0.5 


1.0 


0.3162 


1.5811 



Data Generation Method: Power 

The data generation process used to assess strategies’ power was similar to the data 
generation process used to assess robustness. All variables’ distributions were normal. One 
group’s XY correlation and Y standard deviation were 0.5 and 1.0, respectively, while the second 
group’s XY correlation and Y standard deviation were 0.0 and 0.866, respectively. The XY 
correlations and Y standard deviations across the groups resulted in a population slope difference 
of 0.5 while meeting the nonnality and equal residual variances assumptions of the standard test. 
The population slope difference of 0.5 was selected because when assessed with the four sample 
size combinations considered in this study, strategies' power rates were somewhat closer to 0.5 
than they were to 0 or 1 . Situations where strategies' power rates were not extremely high or low 
produced results that were useful for maximally differentiating the strategies. 

Analysis Strategy 

The assessment of strategies’ robustness involved comparing their average rejection rates 
to the nominal 0.05 rate for conditions where no slope differences existed in the population. 
Deviations from the nominal 0.05 rate were determined to be excessively conservative or liberal 
when they were outside of the two-standard-error band reflective of the number of replications 

used in this study ( 0.05 + /- 2, = 0.046 to 0.054). The standard error band roughly 

V 10,000 

corresponded to Bradley’s (1978) conservative range for robust Type I error rates, 0.045 to 
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0.055. The assessment of strategies’ power involved comparing strategies’ average rejection 
rates to each other for conditions where actual slope differences existed in the population. 

Follow-up analyses were also conducted to gain further insight into how the slope 
estimation strategies were working in the conditions of this study. These follow-up analyses 
included assessments of averages and standard deviations of the strategies’ slope estimates to 
indicate their bias and efficiency, and assessments of strategies’ average standard errors to 
provide understanding of the accuracy of strategies’ significance tests. 

Results 

Type I Error 

Tables 3-9 present the considered strategies’ empirical Type I error rates across 56 of the 
64 combinations of nonnormality, residual variance heterogeneity, and sample size. The 
combination of group sample sizes of 40 and 20 and a residual variance ratio of 1/1 produced 
nearly identical results with the combination of group sample sizes of 20 and 40 and a residual 
variance ratio of 1/1, so the results with sample sizes of 40 and 20 are omitted. Nonnormality 
affected the Standard, James and MM tests similarly, creating liberal Type I error rates when the 
Y distributions were leptokurtic and conservative Type I error rates when the distributions were 
platykurtic. The TW test had Type I error rates that were close to the nominal rate across the 
conditions of nonnormality. The TS test had Type I error rates that were consistently 
conservative across the considered levels of nonnormality. In terms of the hybrid strategies, 
James-TT had Type I error rates that were almost indistinguishable from James, while the James- 
MM, James-TW and James-TS tests had Type I error rates reflective of the nonnormality 
strategy used, being excessively liberal for James-MM, near 0.05 for James-TW, and excessively 
conservative for James-TS. 

The effect of residual variance heterogeneity on Type I error differed for the equal and 
unequal sample size conditions. When sample sizes were equal (Tables 4 and 9), MM was the 
only strategy affected by residual variance heterogeneity, becoming excessively liberal. When 
sample sizes were unequal (Tables 6 and 7), the groups’ sample size-residual variance pairing 
affected the Standard, MM and TW tests similarly, making them liberal with an inverse pairing 
and conservative with a direct pairing. The hybrid strategies were largely unaffected by the 
combination of unequal sample sizes and residual variances. James-TS produced conservative 
Type I error rates for most of the considered residual variance conditions. 
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Table 3 



Empirical Type I Error Rates for Group Sample Sizes of 20 and 20 and a Residual Variance Ratio of 1/1 



Skew 


Kurtosis 


Standard 


James 


MM 


TW 


TS 


James-TT 


Hybrid strategies 
James-MM James-TW 


James-TS 


0 


-1.15 


0.0256 3 


0.0257 3 


0.0260 3 


0.0479 


0.0207 3 


0.0258 3 


0.02 14 a 


0.0456 


0.0204 3 


0 


0 


0.0486 


0.0481 


0.0610 3 


0.0497 


0.0282 3 


0.0483 


0.0486 


0.0479 


0.0262 3 


1.2 


1.11 


0.0668 3 


0.0676 3 


0.1099 3 


0.0508 


0.0304 3 


0.0667 3 


0.0985 3 


0.0490 


0.0358 3 


1.6 


2.86 


0.0941 3 


0.0961 3 


0.1383 3 


0.046 3 


0.0302 3 


0.0949 3 


0.1341 3 


0.0455 3 


0.0345 3 


0 


3 


0.0912 3 


0.0936 3 


0.0999 3 


0.0532 


0.0307 3 


0.0935 3 


0.0874 3 


0.0508 


0.03 17 3 


0 


6 


0.1178 3 


0.1200 3 


0.1226 3 


0.0500 


0.0341 3 


0.1198 3 


0.1091 3 


0.0482 


0.0328 3 


0 


9 


0.1359 3 


0.1403 3 


0.1257 3 


0.0458 


0.0308 3 


0.1395 3 


0.1193 3 


0.0429 3 


0.0288 3 


0 


12 


0.1645 3 


0.1727 3 


0.1347 3 


0.0542 


0.0303 3 


0.1714 3 


0.1343 3 


0.0510 


0.0281 3 



Note. MM = minimum maximum likelihood type, TW = trimmed and Winsorized slope test, TS = Theil-Sen estimator, TT = t-statistic 
transformation. 

3 Type I error rates outside +/- 2 standard errors of the nominal 0.0500 rate (0.0456 to 0.0544). 




Table 4 

Empirical Type I Error Rates for Group Sample Sizes of 20 and 20 and a Residual Variance Ratio of 3/1 



Skew 


Kurtosis 


Standard 


James 


MM 


TW 


TS 


James-TT 


Hybrid strategies 
James-MM James-TW 


James-TS 


0 


-1.15 


0.0428 3 


0.0398 3 


0.0820 3 


0.0566 3 


0.0291 3 


0.0393 3 


0.0346 3 


0.0531 


0.0281 3 


0 


0 


0.0514 


0.0481 


0.1020 3 


0.0511 


0.0323 3 


0.0482 


0.0438 3 


0.0486 


0.0305 3 


1.2 


1.11 


0.0630 a 


0.0598 3 


0.1339 3 


0.0550 3 


0.0304 3 


0.0591 3 


0.0809 3 


0.0509 


0.0292 3 


1.6 


2.86 


0.0792 3 


0.0774 3 


0.1525 3 


0.0517 


0.03 17 a 


0.0757 3 


0.1063 3 


0.0477 


0.0278 3 


0 


3 


0.0689 3 


0.0682 3 


0.1 14 l a 


0.0523 


0.0303 3 


0.0674 3 


0.0653 3 


0.0480 


0.0302 3 


0 


6 


0.093 l a 


0.0946 3 


0.1286 3 


0.0517 


0.0304 3 


0.0938 3 


0.0824 3 


0.0470 


0.0251 3 


0 


9 


0.1106 3 


0.1096 3 


0.1368 3 


0.0481 


0.0326 3 


0.1091 3 


0.0886 3 


0.0440 3 


0.0263 3 


0 


12 


0.1176 3 


0.1206 3 


0.1367 3 


0.0479 


0.0323 3 


0.1202 3 


0.0904 3 


0.0437 3 


0.0235 3 



Note. MM = minimum maximum likelihood type, TW = trimmed and Winsorized slope test, TS = Theil-Sen estimator, TT = t-statistic 
transformation. 

3 Type I error rates outside +/- 2 standard errors of the nominal 0.0500 rate (0.0456 to 0.0544). 




Table 5 

Empirical Type I Error Rates for Group Sample Sizes of 20 and 40 and a Residual Variance Ratio of 1/1 



Skew 


Kurtosis 


Standard 


James 


MM 


TW 


TS 


James-TT 


Hybrid strategies 
James-MM James-TW 


James-TS 


0 


-1.15 


0.0300 a 


0.0306 3 


0.0247 3 


0.0546 3 


0.0263 3 


0.0304 3 


0.0252 3 


0.0532 


0.0259 3 


0 


0 


0.0496 


0.0486 


0.0581 


0.0507 


0.03 14 a 


0.0482 


0.0506 


0.0468 


0.0320 3 


1.2 


1.11 


0.0701 a 


0.0720 3 


0.1174 3 


0.0571 3 


0.0327 3 


0.0713 3 


0.1090 3 


0.0504 


0.0407 3 


1.6 


2.86 


0.1040 a 


0.1054 3 


0.1451 3 


0.0556 3 


0.0321 3 


0.1049 3 


0.1530 3 


0.0526 


0.0412 3 


0 


3 


0.093 l a 


0.0929 3 


0.0981 3 


0.0531 


0.0327 3 


0.0927 3 


0.0870 3 


0.0482 


0.0349 3 


0 


6 


0.1235 3 


0.1286 3 


0.1143 3 


0.0525 


0.0325 3 


0.1269 3 


0.1080 3 


0.0482 


0.0341 3 


0 


9 


0.1524 3 


0.1599 3 


0.1226 3 


0.0522 


0.0371 3 


0.1593 3 


0.1225 3 


0.0485 


0.0356 3 


0 


12 


0.1677 a 


0.1804 3 


0.1253 3 


0.0528 


0.0349 3 


0.1799 3 


0.1293 3 


0.0508 


0.0338 3 



Note. MM = minimum maximum likelihood type, TW = trimmed and Winsorized slope test, TS = Theil-Sen estimator, TT = t-statistic 
transformation. 

3 Type I error rates outside +/- 2 standard errors of the nominal 0.0500 rate (0.0456 to 0.0544). 




Table 6 

Empirical Type I Error Rates for Group Sample Sizes of 20 and 40 and a Residual Variance Ratio of 1/3 ( Direct Pairing ) 



Skew 


Kurtosis 


Standard 


James 


MM 


TW 


TS 


James-TT 


Hybrid strategies 
James-MM James-TW 


James-TS 


0 


-1.15 


0.0134 3 


0.0353 3 


0.025 1 3 


0.0273 3 


0.03 12 a 


0.0352 3 


0.0299 3 


0.0492 


0.0295 3 


0 


0 


0.0218 3 


0.0534 


0.0406 3 


0.0258 3 


0.03 1 3 a 


0.0535 


0.0508 


0.0505 


0.0335 3 


1.2 


1.11 


0.0324 3 


0.0660 3 


0.085 1 3 


0.0274 3 


0.0279 a 


0.0656 3 


0.0913 3 


0.0499 


0.0358 3 


1.6 


2.86 


0.0443 3 


0.0905 3 


0.1 1 7 1 3 


0.0269 3 


0.0345 a 


0.0893 3 


0.1203 3 


0.0488 


0.0383 3 


0 


3 


0.0379 3 


0.0789 3 


0.0654 3 


0.0244 3 


0.0349 a 


0.0792 3 


0.0746 3 


0.0488 


0.0352 3 


0 


6 


0.0612 3 


0.1087 3 


0.0883 3 


0.0283 3 


0.0348 a 


0.1088 3 


0.0953 3 


0.0493 


0.0363 3 


0 


9 


0.0765 3 


0.1327 3 


0.0997 3 


0.0290 3 


0.0347 a 


0.1319 3 


0.101 l a 


0.0466 


0.0344 3 


0 


12 


0.0900 3 


0.1516 3 


0.1 172 3 


0.0294 3 


0.0342 a 


0.1503 3 


0.1 1 5 l a 


0.0486 


0.0286 3 



Note. MM = minimum maximum likelihood type, TW = trimmed and Winsorized slope test, TS = Theil-Sen estimator, TT = t-statistic 
transformation. 

3 Type I error rates outside +/- 2 standard errors of the nominal 0.0500 rate (0.0456 to 0.0544). 




Table 7 

Empirical Type I Error Rates for Group Sample Sizes of 40 and 20 and a Residual Variance Ratio of 1/3 (Inverse Pairing ) 



Skew 


Kurtosis 


Standard 


James 


MM 


TW 


TS 


James-TT 


Hybrid strategies 
James-MM James-TW 


James-TS 


0 


-1.15 


0.0827 a 


0.0360 3 


0.1722 3 


0.1028 3 


0.0305 3 


0.0356 3 


0.0342 3 


0.0497 


0.0337 3 


0 


0 


0.0986 a 


0.0504 


0.1860 3 


0.0989 3 


0.0341 3 


0.0504 


0.0497 


0.0477 


0.0320 3 


1.2 


1.11 


0.1 165 a 


0.0624 3 


0.2039 3 


0.0981 3 


0.0347 3 


0.0615 3 


0.0860 3 


0.0509 


0.0342 3 


1.6 


2.86 


0.1392 3 


0.0798 3 


0.2017 3 


0.0954 3 


0.0386 3 


0.0769 3 


0.0998 3 


0.0503 


0.03 12 a 


0 


3 


0.1359 a 


0.0760 3 


0.1880 3 


0.1007 3 


0.0370 3 


0.0760 3 


0.0682 3 


0.0510 


0.0343 3 


0 


6 


0.1620 3 


0.0997 3 


0.1902 3 


0.1032 3 


0.0354 3 


0.0995 3 


0.0797 3 


0.0531 


0.03 1 1 a 


0 


9 


0.1812 3 


0.1164 3 


0.1879 3 


0.0936 3 


0.0390 3 


0.1148 3 


0.0863 3 


0.0467 


0.0301 3 


0 


12 


0.1862 3 


0.1263 3 


0.1810 3 


0.0887 3 


0.0391 3 


0.1260 3 


0.0903 3 


0.0471 


0.0291 3 



Note. MM = minimum maximum likelihood type, TW = trimmed and Winsorized slope test, TS = Theil-Sen estimator, TT = t-statistic 
transformation. 

3 Type I error rates outside +/- 2 standard errors of the nominal 0.0500 rate (0.0456 to 0.0544). 




Table 8 



Empirical Type I Error Rates for Group Sample Sizes of 40 and 40 and a Residual Variance Ratio of 1/1 



Skew 


Kurtosis 


Standard 


James 


MM 


TW 


TS 


James-TT 


Hybrid strategies 
James-MM James-TW 


James-TS 


0 


-1.15 


0.0280 a 


0.0284 3 


0.0206 3 


0.0526 


0.0289 3 


0.0283 3 


0.0200 3 


0.0526 


0.0300 3 


0 


0 


0.0479 


0.0477 


0.0559 3 


0.0504 


0.0321 3 


0.0479 


0.0498 


0.0501 


0.0353 3 


1.2 


1.11 


0.0672 a 


0.0677 3 


0.1115 3 


0.0484 


0.0388 3 


0.0678 3 


0 . 1101 3 


0.0478 


0.0521 


1.6 


2.86 


0.1028 a 


0.1042 3 


0.1435 3 


0.0495 


0.0371 3 


0.1032 3 


0.1635 3 


0.0494 


0.0483 


0 


3 


0.1006 a 


0.1028 3 


0.0900 3 


0.0518 


0.0391 3 


0.1030 3 


0.0878 3 


0.0518 


0.0428 3 


0 


6 


0.1396 a 


0.1436 3 


0.1044 3 


0.0489 


0.0352 3 


0.1432 3 


0.1057 3 


0.0483 


0.0381 3 


0 


9 


0.1709 a 


0.1752 3 


0.1151 3 


0.0467 


0.0389 3 


0.1743 3 


0.1235 3 


0.0458 


0.0398 3 


0 


12 


0.1952 a 


0.2004 3 


0.1200 3 


0.0483 


0.0396 3 


0.1990 3 


0.1347 3 


0.0476 


0.0387 3 



Note. MM = minimum maximum likelihood type, TW = trimmed and Winsorized slope test, TS = Theil-Sen estimator, TT = t-statistic 
transformation. 

3 Type I error rates outside +/- 2 standard errors of the nominal 0.0500 rate (0.0456 to 0.0544). 




Table 9 



Empirical Type I Error Rates for Group Sample Sizes of 40 and 40 and a Residual Variance Ratio of 3/1 



Skew 


Kurtosis 


Standard 


James 


MM 


TW 


TS 


James-TT 


Hybrid strategies 
James-MM James-TW 


James- 

TS 


0 


-1.15 


0.0360 3 


0.0354 3 


0.0809 3 


0.0506 


0.0352 3 


0.0353 3 


0.03 10 a 


0.0494 


0.0388 3 


0 


0 


0.0494 


0.0479 


0.0884 3 


0.0506 


0.0386 3 


0.0482 


0.0470 


0.0490 


0.0423 3 


1.2 


1.11 


0.0662 3 


0.0636 3 


0.151 l a 


0.0499 


0.0432 3 


0.0639 3 


0.0958 3 


0.0484 


0.0503 


1.6 


2.86 


0.0851 3 


0.0849 3 


0.1828 3 


0.0515 


0.0390 3 


0.0841 3 


0.1236 3 


0.0506 


0.0434 3 


0 


3 


0.0844 3 


0.0821 3 


0.1191 3 


0.0548 


0.0386 3 


0.0820 3 


0.0733 3 


0.0520 


0.0400 3 


0 


6 


0.1025 3 


0 . 1011 3 


0.1279 3 


0.0528 


0.0386 3 


0.1008 3 


0.0820 3 


0.0513 


0.0378 3 


0 


9 


0.1272 3 


0.1290 3 


0.1427 3 


0.0512 


0.0408 3 


0.1278 3 


0.0945 3 


0.0482 


0.0386 3 


0 


12 


0.1426 3 


0.1460 3 


0.1431 3 


0.0486 


0.0367 3 


0.1439 3 


0.0998 3 


0.0474 


0.0322 3 



Note. MM = minimum maximum likelihood type, TW = trimmed and Winsorized slope test, TS = Theil-Sen estimator, TT = t-statistic 
transformation. 

3 Type I error rates outside +/- 2 standard errors of the nominal 0.0500 rate (0.0456 to 0.0544). 




The combination of nonnormality and residual variance heterogeneity (Tables 4, 6, 7, and 
9) produced somewhat unique Type I error patterns for the nine strategies. For the Standard test, 
residual variance heterogeneity usually made the effect of nonnormality less extreme, except for 
when sample sizes were inversely paired with residual variances, in which case Type I error was 
made more extreme. For James and James-TT, residual variance heterogeneity made the effects 
of nonnormality less extreme, though James did not react as much to the combination of unequal 
sample sizes and residual variances as the standard test. The MM test often had the most 
problematic Type I error rates for combinations of nonnormality and residual variance 
heterogeneity. The TW and TS tests were not particularly affected by the combination of 
nonnonnality and residual variance heterogeneity; the TW test was mainly impacted by the 
combination of unequal sample sizes and residual variances, while the TS test was largely 
uninfluenced by anything. The Type I errors of hybrid strategies were reflective of the 
nonnonnality strategy on which they were based, being liberal for James-MM, conservative for 
James-TS, and staying close to the 0.05 level for James-TW. 

Power 

Table 10 compares the power of the nine strategies across three considered sample size 
conditions with normal distributions, equal residual variances, and a population slope difference 
of 0.5. The most powerful strategies were the Standard, James, and James-TT tests, among 
which there was no overwhelming winner. The MM test had smaller power rates than the 
Standard, James, and James-TT tests. The James-MM hybrid test had less power than the MM 
test. The TW and James-TW tests had the smallest power rates of the considered strategies. The 
James-TS and TS tests had higher power rates than the TW and James-TW tests and (mostly) 
lower power rates than the MM and James-MM tests. The use of TS as a hybrid with James 
(James-TS) increased its power relative to the TS test. 
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Table 10 



Empirical Power Rates for Population Slope Differences of .5 and Normality and Residual Variance Assumptions Met 



Sample 


sizes 














Hybrid strategies 




Group 1 


Group 2 


Standard 


James 


MM 


TW 


TS 


James-TT 


James-MM 


James-TW 


James-TS 


20 


20 


0.3910 


0.3901 


0.3735 


0.2270 


0.2655 


0.3906 


0.3372 


0.2215 


0.2891 


20 


40 


0.5096 


0.4989 


0.4730 


0.3082 


0.3931 


0.4994 


0.4403 


0.2930 


0.4202 


40 


40 


0.6909 


0.6912 


0.6359 


0.4369 


0.6179 


0.6912 


0.6180 


0.4353 


0.6483 



Note. MM = minimum maximum likelihood type, TW = trimmed and Winsorized slope test, TS = Theil-Sen estimator, TT = t-statistic 
transformation. 







Slope Estimation 

To gain further insight into the four slope estimation strategies (standard, MM, TW, and 
TS), Table 1 1 summarizes each strategies’ 10,000 estimates of one slope with population value 
0.5 in samples of size 20. When distributions were nonnal, all four strategies gave average slope 
values close to 0.5. The strategies’ standard deviations show that the standard strategy's estimates 
were least variable, followed by the MM estimates, the TS estimates, and finally the TW 
estimates (corresponding to TW’s relatively low power). The strategies' average estimated 
standard errors correspond to the overall liberalness/conservativeness of the strategies’ 
significance tests, and for normal distributions show that on average all methods except for TS 
have standard errors that closely approximate slope variability. TS’s bootstrapped standard errors 
overestimated TS slope variability, corresponding to the conservativeness of its Type I error 
rates. 

The slope estimation results in Table 11 for a leptokurtic Y (kurtosis =12) differ from 
those for a normal Y (kurtosis = 0). For a leptokurtic F, all estimation strategies underestimate the 
population slope value of 0.5, where the least biased estimator is the Standard strategy while the 
most biased is the MM estimate. The Standard strategy's slope estimates are the most variable, 
while the TS estimates are the least variable. The average standard errors of the standard test and 
MM underestimate slope variability, corresponding to the liberalness of the standard test’s and 
MM’s Type I error rates. The TW estimates have standard errors that slightly underestimate 
slope variability. The TS estimate has standard errors that overestimate slope variability, 
corresponding to the conservativeness of TS. The results in Table 1 1 support previous findings 
that the TS estimator is more stable than the MM and Standard estimates when distributions are 
nonnonnal (Wilcox & Keselman, 2004). These results extend previous work by showing that 
with nonnormality, the standard test and MM underestimate slope variability (making the Type I 
error rates of the standard and MM tests liberal), the Winsorized standard errors provide 
relatively accurate estimates of the variability of the trimmed slopes, while the TS bootstrap 
strategy overestimates slope variability (making the Type I error rates of the TS slope test 
conservative). 
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Table 11 



Descriptive Analyses for the Slope Estimation Strategies ( Population Slope = 0.5) 



Skew 


Kurtosis 


Sample 

size 




Standard 


MM 


TW 


TS 


0 


0 


20 


Mean of slope estimates 


0.5043 


0.5043 


0.5079 


0.4998 








Standard deviation of slope estimates 


0.2080 


0.2214 


0.2819 


0.2275 








Mean of slope standard error estimates 


0.2046 


0.2205 


0.2721 


0.2752 


0 


12 


20 


Mean of slope estimates 


0.4598 


0.2987 


0.3538 


0.3280 








Standard deviation of slope estimates 


0.2866 


0.2361 


0.2528 


0.2159 








Mean of slope standard error estimates 


0.1907 


0.1701 


0.2383 


0.2575 



Note. MM = minimum maximum likelihood type, TW = trimmed and Winsorized slope test, TS = Theil-Sen estimator, TT = t-statistic 
transformation. 




Discussion 

In this study, some recently researched strategies for testing independent groups’ 
regression slopes were compared. The standard test of slope differences had robustness problems 
with nonnormality and with the pairing of unequal sample sizes and residual variances. 
Alternative strategies proposed for addressing nonnonnality and for addressing both 
nonnonnality and residual variance heterogeneity were also assessed. The most robust and 
powerful alternative strategies were the Theil-Sen strategy and a hybrid of Theil-Sen and the 
James second-order parametric alternative test. These Theil-Sen strategies had somewhat 
conservative Type I error rates that were largely unaffected by nonnonnality and residual 
variance heterogeneity, and slope estimates that were efficient even for nonnormal data. The 
hybrid strategy of trimming and Winsorizing slope estimates and using them with the James test 
had Type I error rates that were closest of all the considered strategies to the nominal 0.05 level, 
but trimming and Winsorizing also produced slope tests with the lowest power rates of the 
considered strategies. The results also suggest that other strategies are not recommended, 
including James with a test statistic transformation for skewness, MM regression, and the use of 
MM estimates with James. 

To gain some final insight into the considered slope estimation tests, a representative 
sample of 20 observations was generated from this study’s kurtosis = 12 condition. Figure 1 
shows these XY data and plots the Standard least squares regression line. There is one very 
extreme X observation (almost 3 standard deviations from X’s population mean of zero) that is 
also very low on Y (i.e., a bad leverage point). This observation causes the standard slope 
estimation strategy to underestimate the population slope of 0.5 in its slope estimate of 0.421. 
Figures 2 and 3 plot the observations in the data that are not excluded in computing the trimmed 
slope (Figure 2) and the Theil-Sen slope (Figure 3). The trimmed and Theil-Sen strategies 
underestimate the population slope more than the Standard strategy, producing slope estimates of 
0.393 and 0.231, respectively. 
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Figure 1. Least squares regression with nonnormal Y. All 20 observations used to estimate 
the regression line. 




Figure 2. Trimmed means regression with nonnormal Y. Untrimmed slopes used to 
estimate the regression line. 
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X 

Figure 3. Theil-Sen regression with nonnormal ¥. Medians used to estimate the regression 
line. 



Figure 4 is especially useful for understanding the very complicated MM regression. All 
20 of the original observations are used in MM regression, but contribute in weighted fonn to the 
final MM slope estimate. The observations’ weights in Figure 4 show that the high-leverage 
observation is weighed very heavily by MM, causing the MM slope estimate to be relatively 
small (0.148). The observations that are far from the MM regression line are assigned small 
weights. Figure 4 shows that with MM, high leverage points can be weighted such that they 
influence the final slope estimate much more than the Standard least squares estimate. The large 
weights that are assigned to high leverage points in MM result in MM standard errors that 
underestimate slope variability (the W in Equation A5 is large) and inflate the Type I error of 
the MM strategy. Figure 4 makes it clear that the problems of the MM strategy with respect to 
high leverage points are not likely to be fixed by altering the weighting function, £ , or the k that 
determines how each of the scaled residuals are weighted. It may be possible to address MM’s 
problems with high leverage data points through a wise choice of starting values that define the 
MM regression line and the residuals with respect to this line. 
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Figure 4. Minimum maximum likelihood type (MM) regression. Weighted observations 
used to estimate the regression line. 



Implications 

This paper considered only a small number of available slope test strategies. Some of the 
strategies not considered in this paper were excluded because of noted problems and criticisms, 
including nonparametric alternative tests (Deshon & Alexander, 1996; Dretzke et ah, 1982; 
Marascuilo, 1966), residuals-based bootstrapping (Luh & Guo, 2000), ranked data (Headrick & 
Sawilowsky, 2000; Klockars & Moses, 2002), data transformations (Aguinis & Pierce, 1998; 
Glass, Peckham, & Saunders, 1972; Keselman, Carriere, & Lix, 1995; Wilcox & Keselman, 
2004), several robust regression strategies (Anderson & Schumaker, 2003), and judgment-based 
elimination of outliers (He & Portnoy, 1992; Wilcox, 1996). This study suggests that a 
potentially promising slope test strategy would combine the best features of trimming and 
Winsorizing with Theil-Sen. By using the trimming and Winsorizing strategy on the N(N - 1 )/2 
slopes that could be created out of all pairs of observations rather than only N/2 pairs, the final 
trimmed slope estimates should have stability levels that are similar to those of Theil-Sen, 
ultimately improving the power of the trimmed and Winsorized slope test. This proposed test 
would avoid the excessively time consuming bootstrapping associated with the Theil-Sen 
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strategy, reduce the bias of the Theil-Sen estimates for nonnormal data, and provide the analyst 
some flexibility in terms of the extent of trimming used in the final slope estimates. Future 
considerations of how the number of slopes (Ebrahem & Al-Nasser, 2005) and the extent of 
trimming affect Type I error and power for nonnonnal data would be useful for establishing this 
potential test of slope differences. 
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Appendix A 

The Minimum Maximum Likelihood Type (MM) Regression Algorithm 



The first step of MM regression is to obtain robust starting values for the p k 's and o . The 
current SAS procedure for MM uses LTS estimates as starting values (Rousseeuw, 1984; SAS 
Institute, 2003). The basic idea of LTS estimation is to draw samples of K observations from the 
A total observations in the data set. In each sample, obtain least squares estimates of the p k ’s and 



find the ones that minimize ^(e;) , where h 



3N + K + 1 
4 



and observations i through h reference 



the h smallest squared residuals. Additional features of the LTS algorithm involve intercept 

h 2 

adjustments that reduce ^(s;) and computational search processes designed to find final P k 



estimates quickly in extremely large datasets (Rousseeuw & Van Driessen, 2000). One 
preliminary estimate of o is computed as, 

S LTS =d ^Z( S i) ’ ( A1 ) 

where d = 1/. 1 - — ^( 1/c) , c = l/0( ^ + ^ ) , and ® and (/> are the cumulative and probability 
V he 2N 

density functions of the standard normal distribution. 

A more efficient estimate of o than s LTS can also be computed, 




where w, 



0 if |s ; |/s LTS >3 

1 otherwise 
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With initial estimates of the p k ’s and a , the second step is to conduct iterative 



calculations to produce a converged a value, 




(N - K)(J ^(s)50(s)) i 



IN e 9 

ZfH9(<0 



(A3) 



where J 4fs)5®(s) denotes an expected value of c(s) when the s are from a normal distribution 

(about .25 for the Tukey bisquare £(s) with k = 2.9366). In (A3), setting k = 2.9366 results in the 
a having a breakdown rate of 25% (SAS Institute, 2003). 

The third step is to conduct an iterative search for a final solution of the p k 's with a fixed 
o value, 

P m+1 =(x*f!X)' 1 X , aY, (A4) 



d£(s) 1 

where fi is an A by A matrix with diagonal entries — where the s are the scaled residuals 

5s s 

from the mth iteration step and k = 3.44 in default SAS routines (SAS Institute, 2003). The 
entries for f> are the “reweighted” part of MM’s iteratively reweighted least squares algorithm, 
and for the Tukey £ (s) given in (13) are known as the Tukey bisquare weight function. 

At convergence, there are several estimates of the asymptotic variance-covariance matrix 
of p (SAS, 2003). One version is, 



1+ K 


cr 2 (d 2 g(s)/d 2 s) 


(l/CN-K^S^ySe) 2 

i 


N 

V 


"(l/NiXfd 2 ^,)^)! 

Vi ) ) 




(l/N)Xd 2 £(s,)/5 2 8 
V i ) 



(A5) 
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of S, with respect to the residuals, and W is a K by K matrix with entries 

= 2I(a 2 ^(e 1 ) / a 2 e i ) x ik x lk . . 

i 
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Appendix B 

The Critical Value of the James Second-Order Test Statistics 

The significance of the James' test statistic depends on whether it exceeds the following 
critical value, 

James crit = 



c + (1 / 2)(3 * 4 + * 2 )£ [(1 - wjj ) 2 / v j ] 

j 

+(1/16)(3j 4 + j 2 ) 2 [1-(J-3)/ c]^[(l- wjj) 2 / Vj ] 2 

j 

+(1/2)(3j 4 + * 2 )[(8R 23 - 10R 22 + 4R 21 - 6R? 2 + 8R 12 R„ 

-4Rf,) + (z 2 - D(2R 23 - 4R 22 + 2R 21 - 2R^ 2 + 4R I2 R U - 2R 2 n ) 

+(1/ 4)(3j 4 - 2j 2 - l)(4R 12 R n - R 2 2 - 2R 12 R 10 - 4Rf 1 (B 1) 

+4R n Rio - R^ 0 )] + (5 X(, + 2 * 4 + Z 2 X R 2 3 - 3R 22 + 3R 21 - R 20 ) 

+(3/16)(35j 8 +15j 6 + 9* 4 + 5j 2 )(R^ 2 -4R 23 +6R 22 -4R 21 
+R 20 ) + (1/16X9*8 - 3 J 6 - 5 x 4 ~ Z 2 )( 4R 2 i - 2R 22 - R 2 o + 2R 12 R 10 
-4 R „ R 10 + R io) + (1/ 4 )(27 x s + 3* 6 + X 4 + Z 2 )( R n “ R 22 ) 

+(1 / 4X45 X 8 + 9 * 6 + 7x 4 + 3* 2 )(R 23 - R 12 R, t ) 

where Vj = Nj - 2 , c is the 1-a quantile of the central chi-square distribution with J - 1 degrees of 

freedom, R ut = and 2T 2s = — (for * 2 > 2T 4 > X 6 , and Z 8 ,s is 1, 2, 3, and 4, 

' Vj ri(J + 2q-3) 

q=l 

respectively). 
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